Recognizing Plays in American Football Videos

We address the problem of recognizing American Football plays in video. In contrast to recent work on activity recognition this is a much more challenging problem as it involves the actions of multiple players. We propose a method which builds on recent advances in activity recognition, such as using shape and motion based spatio-temporal features and building a space-time representation of the video. Furthermore, we use Multiple Kernel Learning to effectively combine different features. We also propose an extension to the Multiple Kernel Learning method, which optimizes the number of kernels selected, thereby improving efficiency. We demonstrate our approach on a challenging dataset consisting of a variety of football plays and obtain promising results, in the process we also discover some interesting aspects of different types of football plays.